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whether or how a given algorithmic behavior should be addressed. 

If a representation is factually accurate, can It still be algorithmic unfairness? 

Yes. For example, imagine that a Google image query for "CEOs" shows predominantly men. Even if 
it were a factually accurate representation of the world, it would be algohthmic unfairness because it 
would reinforce a stereotype about the role of women in leadership positions. However, factual 
accuracy may affect product policy's position on whether or how it should be addressed. In some 
cases, it may be appropriate to take no action if the system accurately affects current reality, while in 
other cases it may be desirable to consider how we might help society reach a more fair and 
equitable state, via either product intervention or broader corporate social responsibility efforts. 

If a system's behavior is not intended, can it still be algorithmic unfairness? 

Yes. If the behavior is unfair, it meets the definition regardless of the root cause. 


Definition 

“algorithmic unfairness” means unjust or prejudicial treatment of people that is related to sensitive 
characteristics such as race, income, sexual orientation, or gender , 5 through algorithmic systems or 
algorithmically aided decision-making. 


11/29/2017 


Fwd: Fake News-letter 11/27: Efforts to combat spread of (mis/dis)information - Google Groups 


More on misinformation and our role within it. 


- Forwarded message — 

Fromf 

Date: Tue, Nov 28, 2017 at 12:27 AM 

Subject: Fake News-letter 11/27: Efforts to combat spread of (mis dis)information 




Fake News-letter 

BY gTeck Starch - Content 


NOVEMBER 27,2017 INTERNAL ONLY 


SOURCE QUALITY 


Revamping News Corpus 

Goal: Establish “single point of truth” for definition of “news” across Google products. Mitigate risk of 
low-quality sources and misinformation in Google News corpus. 

Status: Define new Google News corpus utilizing existing infrastructure and tools while integrating 
different quality tiers and labels, algo-human content review, new cross-product review/exclusion 
pipeline and updated inclusion Ul. 

CONTACTS: ■■M 
MORE INFO: Proposa 

NEXT STEPS: Research and gather info about possible signals to leverage for product teams to make 
source quality decisions. Finalize and test new rater inclusion guidelines with standards to combat 
misinformation. 
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Definition of Algorithmic Unfairness 



Last Updated 

February 2017 

go/algorithmic-unfairness-definition 

Goals 

Our goal is to create a company-wide definition of algorithmic unfairness that: 

1. Articulates the full range of algorithmic unfairness that can occur in products. This 
definition should be robust across products and organizational functions. 

2. Establishes a shared understanding of algorithmic unfairness for use in the development of 
measurement tools, product policy, incident response, and other internal functions. 1 

3. Is broadly consistent with external usage of the concept. While it is not a goal at this time to 
release this definition externally, it should represent external concerns so that we can ensure 
our internal functions address these concerns. 2 - 3 * 4 

Non-Goals 

The following are not goals for this document: 

1. Specify whether and how Google will take action on potential instances of unfairness that 
involve the use of an algorithm. This will fall instead to product policy. 

2. Describe the consequences of algorithmic unfairness and why they matter 

Definition 

"algorithmic unfairness" means unjust or prejudicial treatment of people that is related to sensitive 

characteristics such as race, income, sexual orientation, or gender, 5 through algorithmic systems or 

algorithmically aided decision-making. 


1 The discussion in Lipton (2016) suggests that instances that present as identical product behavior may upon investigation be 
revealed to have substantially different root causes and therefore require different remediations. Because the nature of an instance 
may not be evident a priori, a wide range of instances are likely to be reported and investigated together. (Zachary Chase Lipton. The 
Deception of Supervised Learning . KDnuggets News 16:n33, September 2016.) 

2 Peter Swire. Lessons from Fair Lending Law for Fair Marketing and Bio Data . Future of Privacy Forum, and presented before the 
Federal Trade Commission Workshop on "Big Data: A Tool for Inclusion or Exclusion?" (2014). 

3 Solon Barocas and Andrew D. Selbst. Big Data's Disparate Impact . 104 California Law Review 671 (2016). 

4 ACM US Public Policy Council. Statement on Algorithmic Transparency and Accountability . January 12, 2017. 

5 This list is not intended to be exhaustive, and additional examples of sensitive characteristics appear in the FAQ. 
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Scope 

The following are in scope for algorithmic unfairness: 

In algorithms that drive predictive systems (e.g., personalization or device financing), this definition 
encompasses automated actions significantly adverse to the interests of a user or group of users, 6 
on the basis of a characteristic that is sensitive in the context of a particular interaction. 

In algorithms that drive representational systems (e.g., search), this definition encompasses the 
implied endorsement of content likely to shock, offend, or upset users sharing a sensitive 
characteristic, or to reinforce social biases. 7 

The following are not in scope for this definition but may be covered by other internal definitions and 
policies: 

1. Biased content for example user-generated content that appears in products (while the 
content itself is not in scope, the algorithmic handling of such content may be in scope) 

2. Insensitive designs which occasion unfairness, and/or interface designs that are less usable 
by groups with sensitive characteristics 

3. Google's internal decisions such as hiring or compensation, which may be influenced by 
algorithms 

Test Cases for Scope 

The following are illustrative examples of what is (and is not) in scope for algorithmic unfairness. Of 
the examples that are in scope, a subset may be determined by product policy to require 
remediation. 

Google Display Ads for High-Paying Jobs 

CMU published a study in 2015 with experiment-based observations, arguing that Google's ad 
serving system perpetuates gender bias on the basis of two campaigns that were found to target 
high salary jobs at male users on the Times of India website. 8 The effect was highly sensitive to the 
ads from this particular service, and the same effect was not reproduced in several other 
experiments by the same authors. The cause was found to be higher CPA (cost per conversion) for 


6 This includes effects which are small for a single instance but have a significant cumulative effect. As Greenwald et al. observe, 
statistically small effects can have substantial societal impact when they apply to many people, or if they apply repeatedly to the 
same person. (Anthony G. Greenwald, Mahzarin R. Banaji, and Brian A. Nosek. Statistically small effects of the Implicit Association 
Test can have societallv large effects . Journal of Personality and Social Psychology, 108:553-561, 2015.) 

7 Note that even small reinforcement biases in systems can be magnified via positive feedback loops, since biased representations 
can influence human behavior which is in turn fed back into training data for those systems. 

8 Amit Datta, Michael Carl Tschantz, and Anupam Datta. Automated Experiments on Ad Privacy Settings . PETS 2015, pp. 92-112, 
June 2015. 
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female users for one campaign (notably with a higher CTR for female users) and advertiser targeting 
to male-only users for the other. 9 

In scope for algorithmic unfairness. In the first case, bias in user behavior trained the system to have 
biased targeting. In the second case, advertiser-selected criteria were related to a sensitive 
characteristic. 

Facebook Computation of Unregulated FICO Scores 

Researchers raised concerns in 2015 about Facebook computing non-regulated credit scores based 
on user activity. 10 One substantial issue they commented on is that credit scores are regulated by 
the Equal Credit Opportunity Act of 1974, which prohibits creditors from discriminating against 
applicants on the basis of race, religion, national origin, sex, marital status, age, or receiving public 
assistance; however, the non-regulated scores did not appear to have such restrictions. 11 

In scope for algorithmic unfairness. Based on the user's behavior, the system automatically 
computed sensitive characteristics that could adversely affect their financial opportunities. 

Chumhum Suppression of Businesses in High-Crime Areas 

An episode of the primetime television drama "The Good Wife" featured a tortious interference case 
against a fictional search engine company (Chumhum) for releasing a maps application that 
suppressed businesses in high crime areas. 12 

In scope for algorithmic unfairness. Content was excluded by an algorithm in a way that 
disproportionately affected people who owned businesses in neighborhoods correlated with the 
sensitive characteristics race and income. 

West African Spam Filters 

An external researcher poses the following: "One question is whether the design of spam filters 
could make certain individuals more susceptible to having their legitimate messages diverted to 
spam folders. For example, does being located in a hotbed of Internet fraud or spam activity, say 
West Africa (Nigeria or Ghana) or Eastern Europe, create a tendency for one's messages to be 
mislabeled as spam?" 13 

In scope for algorithmic unfairness. In this hypothetical, the system learns associations and 
downgrades or excludes content related to sensitive characteristics such as national origin and 
race. 


9 Giles Hogben, Alex McPhillips, Vinay Goel, and Allison Woodruff. Allegations of Algorithmic Bias: Investigation and Meta-Analvsis . 
September 2016. oo/alleoations-of-aloorithmic-bias 

10 Tressie McMillan Cottom. Credit Scores. Life Chances, and Algorithms . May 30,2015. 

11 Astra Taylor and Jathan Sadowski. How Companies Turn Your Facebook Activity Into a Credit Score: Welcome to the Wild West 
of Data Collection Without Regulation . The Nation, May 27, 2015. 

12 The Good Wife, Episode 'Discovery'. CBS, first aired November 22,2015. 

13 Jenna Burrell. How the machine "thinks': Understanding opacity in machine learning algorithms . Big Data & Society 3(1):1-12, 
2016. 
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Google Photos Use of Gorillas Label 

In June 2015, a web developer posted on Twitter that Google Photos had tagged an image showing 
him and a friend at a concert with the label "Gorillas". "Of all terms, of all the derogatory terms to 
use," Alcine said later, "that one came up." According to a post mortem, Google executives noticed 
Alcine's tweets within one hour. A Googler reached out through Twitter for permission to access the 
user's photos, and the issue was identified and resolved. In the short term, the Photos team stopped 
suggesting the "Gorillas" tile in the Explore page and stopped showing Search results for queries 
relevant to gorillas; in the long term, the team pledged to investigate the image annotation models 
that generate false positives for the gorilla label and work on the image annotation pipeline and 
quality evaluation processes. 14 

In scope for algorithmic unfairness. The system drew an incorrect inference that is offensive to 
members of a given race. 

Autocomplete Results for Trayvon Martin 

In 2013, Autocomplete results showed negative results for Travyon Martin (e.g., "drug dealer"), but 
more positive results for George Zimmerman (e.g., "hero"). 15 

In scope for algorithmic unfairness. The system processed and showed offensive content from 
users, in a way that could be seen to reinforce existing social biases regarding race. 

Google Search Results for Black Girls & Pornography 

In 2013, a researcher expressed concern that search queries on Google for "black girls" historically 
yielded a high percentage of pornography. 16 

In scope for algorithmic unfairness. The system showed results that are offensive and reinforce 
existing social biases regarding race. 

Google Image Search Results for Physicists 

Google image search results for "physicist" show predominantly men. In reality, roughly 20% of 
physicists are women. First, imagine that the image search results show 1% women. Second, 
imagine instead that the image search results show 20% women. 

In scope for algorithmic unfairness. In the first case, when the image search results show only 1 % 
when the reality is 20%, the system is amplifying an existing bias in society. In the second case, when 
the image search results show a percentage similar to the current reality, the system is reflecting an 


14 Giles Hogben, Alex McPhillips, Vinay Goel, and Allison Woodruff. Allegations of Algorithmic Bias: Investigation and Meta-Analvsis . 
September 2016. go/alleoations-of-aloorithmic-bias 

15 Safiya Umoja Noble. Trayvon. Race. Media and the Politics of Spectacle. The Black Scholar. 44(1 ):1 2-29, Spring 2014. 

16 Safiya Umoja Noble. Google Search: Hvper-visibilitv as a Means of Rendering Black Women and Girls Invisible . Invisible Culture: 
Issue 19, October 29.2013. 
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existing bias in society. Both cases fall in scope for algorithmic unfairness because they reinforce a 
stereotype about the role of women in a scientific discipline. However, while both are in scope for 
algorithmic unfairness, remediation may be more likely in the former case. (As with the other cases, 
whether and how to remediate would fall to product policy.) 

Microsoft Kinect 

News media in 2010 and 2017 reported on developers and users of Microsoft's Kinect experiencing 
difficulties with facial recognition or motion detection for users with dark skin in low light conditions. 
Affected users could not use certain system functions including automatic sign in to the user's 
profile, 17 or use software that required motion detection to function. 18 

In scope for algorithmic unfairness. The predictive system performed poorly for users with dark skin, 
which is associated with the sensitive characteristic race. Users considering purchasing the Kinect 
or software that requires the Kinect would be unlikely to suspect that their skin color might make it 
difficult or impossible to use advertised features (such as automatic login using facial recognition, 
gestural menu interactions, motion controlled gaming). Gaming hardware and software purchases 
are commonly non-refundable after being opened, so a user who realized the Kinect or its software 
could not track them would be left with a non-functional product they could not return. 


Survivalist Game on the Play Store 

In January of 2016, a series of Twitter posts and a Change.org Petition raised concerns about a 
Survivalist game on the Play Store, claiming it gamified killing aborigines. 

Not in scope for algorithmic unfairness. There were concerns that the content was offensive, but an 
algorithm was not involved. 

Airbnb Acceptance Rates 

Researchers from the Harvard Business School conducted an experiment on Airbnb and found that 
applications from guests with distinctively African-American names were less likely to be accepted 
relative to identical guests with distinctively white names. The study could not identify whether the 
unfairness was based on race, socioeconomic status, or some other factor. 19 

Not in scope for algorithmic unfairness. The unfairness was on the part of individual users ("sellers" 
on Airbnb) and there was no algorithmic component to the unfairness. While this is out of scope for 
algorithmic unfairness, Airbnb did however recognize that their design choices unrelated to 
algorithms could be facilitating unfair behavior. Accordingly, it pursued non-algorithmic 
remediations, such as modifying its user interface design and its end user policy. 20 


,7 Brendan Sinclair. Kinect has problems recognizing dark-skinned users? Gamespot, 2010. 

18 Andy Trowers. How We Accidentally Made a Racist Videogame . Kotaku, 2017. 

19 Benjamin Edelman, Michael Luca, and Dan Svirsky. Racial Discrimination in the Sharing Economy: Evidence from a Field 
Experiment . Harvard Business School NOM Unit Working Paper (16-069), 2015. 

20 Laura W. Murphy. Airbnb's Work to Fioht Discrimination and Build Inclusion: A Report Submitted to Airbnb . September 2016. 
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FAQ 

Is personalization the same as algorithmic unfairness? 

Generally, no. Personalization is algorithmic behavior that presents different results to different 
users. Personalization is very often beneficial to users and is not necessarily unfair. In fact, 
personalization may sometimes remediate unfairness by identifying an individual's interests more 
precisely than would the use of a sensitive characteristic. 21 However, in some cases personalization 
may be strongly associated with a sensitive characteristic in a way that causes significant harm to a 
user or users, in which case it would be algorithmic unfairness. 

Can any characteristic be sensitive? 

No. For example, "people who like yellow" or "people with pets" are unlikely to be sensitive. 22 

Which characteristics are sensitive? 

Sensitive characteristics may be determined by legal considerations or by more broad principles. 
Legally determined characteristics may vary by jurisdiction, sector, or other factors. Sensitive 
characteristics determined by more broad principles are especially likely to include characteristics 
that are associated with less privileged or marginalized populations (particularly when such 
characteristics are immutable), may be socially undesirable, or may be associated with civil liberties. 

Specific examples may include race/ethnic origin, gender identity, sexual orientation, religion, 
political party, disability, age, nationality, veteran status, socioeconomic status (including caste and 
homelessness), and immigrant status (including refugee and asylum seeking status). Further, there 
may be emergent sensitive characteristics for which we do not yet have a name, or which we have 
not yet anticipated. 23 

Are proxies for sensitive characteristics covered? 

Yes. Proxies are characteristics that are highly correlated with a sensitive characteristic. Actions 
based on close proxies for sensitive characteristics are in scope for further investigation. For 
example, if interest in hip-hop music is highly correlated with being Black, targeting users who like 
hip hop music may result in algorithmic unfairness based on race, regardless of intent. 

If a system's behavior is caused by societal bias, can it still be algorithmic unfairness? 

Yes. Societal bias that is reflected in algorithmic behavior is a central issue in the external public, 
media, and regulatory concerns about algorithmic unfairness. 24 - 2526 ' 27 Additionally, one often doesn't 


21 James C. Cooper. Separation and Pooling . George Mason Law & Economics Research Paper No. 15-32, March 2016. 

22 http://civilriqhts.findlaw.com/civil-riQhts-overview/what-is-discrimination.html 

23 danah boyd, Karen Levy & Alice Marwick, The Networked Nature of Algorithmic Discrimination . In Seeta Pena Gangadharan, 
Virginia Eubanks, and Solon Barocas (eds). Data and Discrimination: Collected Essays. Washington, D.C.: Open Technology Institute, 
New America Foundation, pp. 53-57,2014. 

24 Tarleton Gillespie. The Relevance of Algorithms . In Media Technologies: Essays on Communication, Materiality, and Society, 
edited by T. Gillespie, P. Boczkowski, and K. Foot. MIT Press, Cambridge, MA, 2012. 

25 Solon Barocas and Andrew D. Selbst. Big Data's Disparate Impact . 104 California Law Review 671 (2016). 

26 Executive Office of the President. Big Data: Seizing Opportunities. Preserving Values . May 2014. 
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know the root cause of algorithmic unfairness without an investigation, so any real-world process 
(e.g., incident response) is best served by encompassing a wide range of potential causes. 
Accordingly, this definition covers a wide range of sources, from machine learning classification 
errors to societal bias. However, the nature of the root cause may affect product policy's position on 
whether or how a given algorithmic behavior should be addressed. 

If a representation is factually accurate, can it still be algorithmic unfairness? 

Yes. For example, imagine that a Google image query for "CEOs" shows predominantly men. Even if 
it were a factually accurate representation of the world, it would be algorithmic unfairness because it 
would reinforce a stereotype about the role of women in leadership positions. However, factual 
accuracy may affect product policy's position on whether or how it should be addressed. In some 
cases, it may be appropriate to take no action if the system accurately affects current reality, while in 
other cases it may be desirable to consider how we might help society reach a more fair and 
equitable state, via either product intervention or broader corporate social responsibility efforts. 

If a system's behavior is not intended, can it still be algorithmic unfairness? 

Yes. If the behavior is unfair, it meets the definition regardless of the root cause. 

If unfairness is executed by an algorithm but is the result of a human decision, can it still be 
algorithmic unfairness? 

Yes. For example, if a human chooses unfair keywords for ads and those choices result in unfair 
algorithmic choices of what ads to show to a user or users, that would fall within the scope of 
algorithmic unfairness. 

Does this definition include the use of data by multiple parties? 

Yes. For instance, decisions which may result in data being provided to third parties via APIs, being 
sold to third parties, or acquired via federated identity should consider the possibility that that data 
could be used to power unfair algorithmic decision-making. 

What is the relationship between this definition of algorithmic unfairness and the fairness measure 
in Hardtetal. (2016)? 

The fairness measure in Hardt et al. (2016) 28 is a statistical guarantee that, in general, members of 
one category are classified with the same accuracy as members of another category. For example, if 
Black people who apply for credit cards are classified with lower accuracy than white people (e.g., 
Black people who would actually repay their loans are wrongly classified as bad credit risks), the 
fairness measure in Hardt et al. (2016) would find that to be a problem. However, if most people who 
happen to have a particular sensitive characteristic are correctly classified as being very likely to 
default on their credit cards (regardless of whether that classification is done based on that 
sensitive characteristic or based on some other associated variable), the Hardt et al. (2016) measure 
would not detect it as an issue. Therefore, the fairness measure in Hardt et al. (2016) statistically 
tests for a certain type of machine learning classification error that would constitute a specific type 
of algorithmic unfairness, encompassing some but not all of the cases covered by the current 
definition. 


27 ACM US Public Policy Council. Statement on Algorithmic Transparency and Accountability . January 12, 2017. 

28 Moritz Hardt, Eric Price, and Nathan Srebro. Equality of Opportunity in Supervised Learning . arXiv.org, October 2016. 
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SOURCE QUALITY 


Revamping News Corpus 

Goal: Establish "single point of truth" for definition of "news" across Google products Mitigate risk of 
low-quality sources and misinformation in Google News corpus 

Status: Define new Google News corpus utilizing existing infrastructure and tools while integrating 
different quality tiers and labels algo-human content review^ new cross-product review/exclusion 
pipeline and updated inclusion UL 

rrm ta r_T s 

MORE INFO: PfSposa Udate 3 j Spate (WIP) / Stress Test publisher list 

NEXT STEPS: Research and gather info about possible signals to leverage for product teams to make 

source quality decisions. Finalize and test new rater inclusion guidelines with standards to combat 

misinformation. 


Trust Project & Nutrition Labels 

Goal: Develop transparency standards that help people easily assess quality and credibility of 
journalism Work across News Nutrition Labels and Search Console / 3F structured data teams to 
incorporate Trust Project data into future product plans 

Status: Launched eight trust indicators with a dozen publishers going live with schema 
implementations. Launched publisher Knowledge Panels in Search 

MORE INFQl^uDlisnS^flaunc^^HnouncemSnnne Trust Project 

NEXT STEPS: Complete PRD Addendum and SD Playbook for markup support in Search Console 


Reviewing Rater Quality 

Goal: Evaluate general performance of Ewok rater evaluations in select countries for News corpus inclusion to 
uncover potential ted flags or unusual patterns. 


https://cpoci ps.goog le .com' a g oogle .comfforum ^msg g oog le ■ sos -e ng .'k JFIQ21C ww4/D ^ -1 bXkBQAJ 
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Fwd: Fake Newsletter 11/27: Efforts to combat spread of (mis/cfsjinfonTialjori - Google Groups 


Status: Analyze low-quality and high-quality rater comments, examine inconsistencies and rater 
inclusion history. 


CONTACTS 



MORE INFO: A/C Privileged and Confidential 


PRODUCT OPERATIONS 


Project Purple Rain: Crisis Response & Escalation 

Goal: Establish and streamline news escalation processes to detect and handle misinformation across 
products during crises. Install 24/7 team of trained analysts ready to make policy calls and take 
actions across news surfaces including News News 360 and Feed. 

Status: SOS Alerts. Crisis Response. HotEvent. and T&S Incident Management teams are 
collaborating to identify a narrow set of queries that would be used to manually trigger flight-to-quality 
in Search. T&S Incident Management team is currently looking to expand and share resources with 
teams that currently handle Suggest and Web Answers escalations. 

MORE 


Policy Development & Enforcement 

Goal: Develop and proactively enforce mis formation policy that scales across News products. Explore 
cross-product toolset for removal enforcement. 

Status: Drafted policy for Coordinated Inauthentic Cross-Border Information Operations (CiCBIQ). 
Reviewed proposed policy with Search & News leadership 

contacts 

MORE INFO^ra^TOposan^xpan^onc^ntorTemen^™ 

NEXT STEPS: Set up workflows and tools with TAG team and other stakeholders. jeffreyc@ and 
@jacobhelberg@ to scope “badness baseline rating' and other tools for top of P0 escalation funnel. 


ESCALATIONS 


Italian sites spread misinformation: PR escalation following Buzzfeed article. 3 sites rejected 
following ewok and manual evaluations. 

Hoax science stories in News: Junk science articles flagged by internal Googlers and written up in 
blog. Status pending: 5 sites submitted for ewok re-evaluation, 3 rejected 

Dylann Roof sentencing resurfaces in News: Danny Sullivan's sleuthing uncovered cause as 
publisher error that spawned copycats (Ftesolved on Twitter) 

CA shooting rampage: No major issues across pm ducts PR post mortem with open questions and 
follow-up here, 

TX church shooting: We surfaced some misleading tweets in Search and some low-quality videos in 
YouTube which resulted in a negative press cycle. Danny Sullivan responded with a series of tweets. 
(PR summary) 


https:// groj ps.goog le .com ag aogfe.canffbnjm/ftinisg/g aag le-sos-eng/k JFtQ21 C ww4Z01 -1 bXkBQAJ 
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Expanding Collaboration 
for News Quality 

Fatih Ozkosemen, Trust & Safety Search 
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Global Partnerships 


Google efforts to address 
Fake News 

What Google is doing to address the problem 

ao/FakeNews 

Gustavo Fuentes-Rivera, Pauline Peyronnet, Mark Lyall 
Dec. 2016 
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Google 



o 

Users have access to a large variety of news 
sources 


Google 



The web offers a large 
variety of options. 

From reputable, traditional 
news sites to newer, 
relatively unknown news 
outlets. 
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Many users access news sites via FB News Feed 


o 

Global Partnership* 
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but some bad actors came into play 


o 

Global Partnerships 


sun yum mmut- 

TRENDING ® Hit 2) & @ 

BREAKING: Fox News Exposes Traitor Mcgyn 

y* Megyn Kelly 

61K people talking about this 

Kelly, Kicks Her Out For Backing Hillary 

a * Anthony Weiner 

Megyn Kelly has brought a lot of heat on her many endeavors to blemish 

13K people talking about this 

OOP presidential candidate Donald Trump, and tha Fox News viewers are 

zv* Beyonce 

not satisfied Tha Kietly File’s raongs have avan falen behind Maddow 

41K people talking about this 

Show on MSNB. which is dsgraceful VIA National Insider Politics Via: 

Governor of North Dakota 

Conservati 

4.7K people talking about this 

endingthefed com 

a * Don Cheadle 


11K people taking about this 

Posts by Gregory M. Lea, Chris Conard and 13 hours ago 

a * My Ian 

others... 

290K people talking about this 


In the context of the US elections, a number of bad actors used the 
Facebook News Feed to promote fake news sites 


Google 


And the issue gained notoriety 
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Tech fe Chart of the Day 

Fake News Is A Real Problem 

Facebook engagement of the top five fake election stories’ 


Headlvw 

"Pope Francis Shocks World. Endorses Donald 
Trump for President. Releases Statement* 

Ending in* Fad 

"Wiklleaks CONFIRMS Hillary Sold Weapons to 
ISIS ...Then Drops Another BOMBSHELL! Breaking News' 

Tha PoMKai lewder 


tngugamanu 


960,000 


"ITS OVER: Hillary's ISIS Email Just Leaked & It s 
Worse Than Anyone Could Have Imagined* 

Ending the rad 

"Just Read The Law: Hillary Is Disqualified From 
Holding Any Federal Office* 

Ending ttia Fad 

"FBI Agent Suspected in Hillary Email Leaks Found 
Dead In Appartment Murder-Suicide* 

Oanvar Guardian 


7S4.000 


701.000 


S67.000 


Total Facebook engagement for top 20 election stories (August-election day) 

Fake news 8.7 m 

Mainstream news 7.3 m 
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We discovered that some of these sites were 
using AdSense to monetize their traffic 
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USA TODAY 
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Google to pull ads from fake news sites 


SAN FRANCISCO — Amid criticum that fake newt 
tile* have contributed to mamtormation and 
increasing polarization among the American public. 
Google announced Monday night that A wil stop 
allowing fako newt tilet to ute its ad software 

Facebook followed with a similar pokey ‘In 
accordance with the Audience Network Pokey, we do 
not integrate or display ads m apps or sites 
containing content that Is ikegal. misleading or 


and we took quick action, 
updating our AdSense 
policies to address 
'Misrepresentative content' 


Gooqle lnternal 0nl 
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O 

With this update we disallow sites that mislead 
users 


Approved Public Statement: IVeVe updated our publisher policies and will start prohibiting Google ads from 
being placed on misrepresentative content; just as we disallow misrepresentation in our ads policies . Moving 
forward, we will restrict ad serving on pages that misrepresent, misstate, or conceal information about the 
publisher, the publisher's content, or the primary purpose of the web property. 


This update solved the problem of fake news sites monetizing with AdSense. 
We're working on a similar update for AdX to cover all our bases. 

Additionally there is a working group established to assess whether changes 
are needed for our platform products (DBM, DFP) 
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But then another problem surfaced: Ads 
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Ellen’s Goodbye SHOCKS Fans! There’s No Joy For The View 



We received escalations 
from publishers about ads 
promoting misleading 
information/sites. 
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Proprietary ♦ Contidenbal 



How are we addressing this? 


(i) Reviewing suspect ads and taking action in cases where 
there is a violation of our ads policies. 

(ii) Assessing what improvements can be done in automation 
support and/or review procedures 

(iii) Preparing guidance for publishers on what steps can they 
take on AdX to block ads that they consider undesirable 


Google 


Internal Only 

Proprietary ♦ Confidanbal 







Resources for external communications 



“Misrepresentative Content" update on AdSense Policy - Comms Doc 


Approved Public Statement: WeVe updated our publisher policies and will start prohibiting Google ads from 
being placed on misrepresentative content, just as we disallow misrepresentation in our ads policies . Moving 
forward, we will restrict ad serving on pages that misrepresent, misstate, or conceal information about the 
publisher, the publisher's content, or the primary purpose of the web property. 


AdSense Program Policies - Prohibited content 


Misrepresentative content 

Whafs the policy? 

User* don't want to be misled by the content they engage with online. For thla reason. Google ads 
may not be placed on pages that misrepresent, misstate, or conceal information about you. your 
content or the primary purpose of your web property. 

Examples v 
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Google 


Proprietary ♦ ConlldenSal 
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Additional resources 


Internal FAQ for AdSense Misrepresentative Content Policy 


FAQ on "Fake News" 

Q: What is Google's policy on "fake news ’? 

A: Google does not have a specific policy related to "fake news". "Fake news" is not a clearly 
defined concept and any policy related to it would require Google to make the assessment as to 
what is the truth around the world. Google does have key policies that protect users from 
misrepresentative content and hate speech which are addressing the badness being seen on 
the network. 
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Summary 

Two key areas of concern: 


1. Monetization 

What is Google doing: 

We have updated our AdSense policies to restrict ads on sites that 
misrepresent, misstate, or conceal information about the publisher. 

To cover all bases we will make a similar update to AdX policies. 

2. Ads 

What is Google doing: 

> We are reviewing suspect ads and taking action in cases where there is a 
violation of our ads policies. 

> We are assessing what improvements can be done in automation 
support and/or review procedures 

> We are preparing guidance for publishers on what steps can they take on 
AdX to block ads they consider undesirable 

Google 
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Beyond 
Web Spam 


Increasing policy coverage over 

• Abuse 

Other spam types, malware, deception etc. 

• Fringe/controversial 

Factually incorrect, fake, irrelevant, non-news etc. 

• Sensitive 

Hate, geo-politically sensitive, diversity & bias etc. 

• Inappropriate 

Sexually explicit, graphic violence and vulgarity etc. 



T&S 
can help 


• Badness signal discovery & data sharing for policy 
enforcement 

• Multi-lang market analyst support across all top 
languages globally 

• Leveraging TVC resources where automation is not 
possible 

• Aligning with policy (GPP is part of T&S) 

• T&S Eng resources 


Proposed 

Als 


• setting up a metric for corpus quality 
News corpus badness rate 

• broader policy enforcement and abuse detection 

Beyond webspam 

• better detecting low quality 

lq sources + Iq content in hq sources 

• clean & regularly sanitized news corpus 

both for site registration + ongoing 


































Isn’t it curious Now nearly every person’s head 
falls along roughly the same horizon line? 


-.MriW VJjiS ; 








It's the result of the photographer being 
average height. From an outlier’s perspective, 
the world might look significantly less uniform. 












This is a talk about 
the role of humans in 
machine learning. 





But really it’s a talk about 
the role of humans in 
decision making. 



"It's true that they con follow instructions at superhuman speed, with 
superhuman fidelity Gna over unimaginable quantities of data. But these 

instructions don't come from nowhore. Although na.roi networks might 
be said to write, their own programs, they do so towards goafs set by 
■ humans, using data collected for human purposes . If the data is skewed L 
even by accident, the computers will amplify injustice," 


— Hie Guardian 





. “ft/s true that they con follow ‘instructions at superhuman speed, with 
superhuman' fidelity and oyer unmagnafye quantities 'of data But these 

instructions don't ccrmo from ho where,Although nc-uraf networks'might 
be s aid tb v/rife their' own ■programs they do so towards qoaSs set by 
humans, usingdao cotibcted for hurtanpurpop**^ ne nany^skr-wiyi. 
even by accident, the computers w( amplify injustice. J 


■ — The Guardian 








Recidivism 
software is 
biased against 

black people 


Photo-editing 
app makes faces 
look more 


Caucasian 


Researchers 
claim facial 
attributes predict 
criminality 


Machine Oias fProPublcol 


FaceAuu quo o^izca for building j 
racist AI fTcchCrunchl 


Mn c flxnmplEfi con fofl foinxf on tic Blui fc.i nfirp far f/n rhinc 


PhvSiOQr'iomv's New Clothe:,. 
(Medium) 




When we presume that h jman judgment can—or 
should—be removed from the loop, the result Is an 

unconscious bias network effect 


And we (Googlers) are iustas susceptible to this effect as our users, 









Training data are 
collected and 

classified 


Algorithms a re 
programmed 







Media are 
filtered, ranked, 
aggregated, or 
generated 








Training data are 
collected and 

classified 


Algorithms are 
programmed 



Media are 
filtered ranked, 
aggregated! or 
generated 


People {///re us) 
are programmed 




Unconscious bias gets reinforced in the training cats 






Media are 
filtered, ranked, 
aggregated, or 
generated 
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Unconscious bias affects the way we collect and classify data, design, and write code 








If we are going to make the world's information 
universally accessible and useful we must st F ive to make 
the products we are developing—apps* infrastructure* 
models* and more— work for everyone. 




If we are going to make the world’s information 
universally accessible and useful, we must strive to make 
the products we are 
• ; models, and mor 





Leareimortal uu ■■■ 'hSm-ss 



And unfortunately, we (humans) have a 
history of making product design 
decisions that are in line with defaults , 
and not with the needs of everyone. 


' For the- pur-pose* hi * taiK tno te!m "defcun" io repre^ni ch? irupicl adHuflos =? 0.1 Hu re cr eoc lety 
tiorarally oJb#cv 1 b*b to. VptcdiVy withoutCDtiulou^ raararMBfl. M010 ln*omiHJMi con kofaund 1 



Female drivers are 47% more likely to be severely 
injured in an auto accident because, until 2011 
female body-type crash test dummies weren't 
required by the U,S, Department of Transportation, 
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Until the 1990’$, virtually all color calibration *or color film 
was based on Caucasian skin tones. It wasn’t until Kodak 
started receiving complaints from cftocolatiers and wood 
furniture manufacturers that they began to invest research 


into supporting a broader range of brown tones. 


BJKE1S 
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In the top 100 grossing U S. live-action 
films from 2014-2016, male characters 
were seen and heard nearly twice as 
often as female characters. 




Where are these 
disconnects coming from? 




Are automakers 
intentionally trying to 
harm women? 


Was the decision to 
color balance for light 
skin overtly racist? 


Are creatives in the film 
industry marginalizing 
women on purpose? 




These are often the behaviors of rational 
actors making whatsee/nec/ like ’obvious’ 
choices, without malice or ill-will. 






Take for example the 
case of “David” 







Or was it “ 









For the purposes of this study, it was both* 


Participants watched a series of videos of babies. 
For each video, half of the participants were told 
a boy’s name, the other half a girl’s name. 



When the babies did something 
unambiguous, reports were not 
affected by the perceived gender, 

if the baby dearly smiled, for example, everyone 
said the baby was smiling or happy. 
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Then the babies played with a 
jack-in-the-box toy. When itsuddenly 
popped up, the child was startled 
and jumped backward. 


SHIKES 
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She's afra d” 


He’s angry 






And most of us would agree that 
it would be irrational behavior to 
treat a fearful child the same way 
we’d treat an angry child. 



it knowledge o ta cnaa 's gender effects octets' perception of 
that child then mate and female children nr& going to elicit 
different reactions from the world, different patterns of 
encouragement These perceptions matter ; ei/e/i in parents 
wf\o ore committed to treating sons orrJ daughters alike/* 

— Eliza belli Spclkc 
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Human perception drives 
virtually every facet of 
machine learning. 



I propose we* make machine-learning 
intentionally human-centered and 
intervene for fairness. 


- vug =Htiman5. fnis IehiI simething anyone canpen* should betftfng in isolation, but we're in a goal positonio sterl. 







Tenets 

Designing for fairness 
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Be 

Accountable 


We can’t take our hands 

off the steering wheel. 

In rejecting the myth of neutral date, 
we are committing to be more 
conscious and conscientious. 
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Be 

Accountable 


Present day 
U R obots 

i* Yada yad a y a da 
u F utur e 

Present day 

U Humans 
U Still humans 
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Be 

Skeptical 


Challenge assumptions at 
every turn. 

We can't blindly rely on the systems 
that underpin conventional wisdom. 



Journalism 

Fake novjs is indislinaLishflhlo 



Be 

Skeptical 


Customer reviews 

M<ile reviews skew average scores 


Medical science 

Experiments pygr-rscruh 


Standardized tests Crime statistics 

SAT sco.es don't predict grades Racial twofillno Is real 





Be 

Humane 


Success metrics should 
bring out the best In 
human nature. 

Standard engagement metrics confine 
people to whatever they’ve done 
before, rather than empowering what 
they’re capable of doing next. 
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Be 

Humane 


Time well spent 

tinroivoilspcnt.uQ 

Learning and expression 
Exploration and connection 
User-defined goals 




Be 

Humble 


We don’t always know 
better. 

The tech industry is in love with 
“disruption”, but frequently that 
means imposing a vision of the 
future onto users and expecting 
them to adapt. 



Be 

Humble 


“There are two ways to get people 
used to automation. The soft and 
fuzzy way. ..Just keep reassuring 
people until they’re comfortable . 
And then there's the second way: 
let the humans take control when 
they need to ." 


— NFR 





Augmenting - not _ Automating 
Supporting - not -* Replacing 




In practice 


An ML user journey in 12 phases 



We come up with the ideas 

What will the model actually preetfcf? 
Who stands to benefit the most from It? 




We define the ML strategy 


Why would heuristics be less effective 
than a machine-learned model? 





We conduct the market research 

How do users describe :he benefits of ML? 
What reference points are tliey bringing? 





We design the experiences 

How are people solving these problems today? 
How might ML Improve things? 





















We design the Rater protocols 

Hypothetically speaking. would this be an 

unambiguous task Tor end-users to perform? 


{Author snate: While me L'se c>f Raters is primori.y for supervisee! earning, labeled data—grautd trnlh or otherwise—have o wide 'variety 
otoppl'totlcri. so 1 eftose io ifletode this as o pfomftiefli jofasc. fto'rtfforcimeflf icarntns is likely (ft artfy place ffrej/Vc cnarofy trfcscflL 




Sidebar on Raters 


Speed arid Agreement are the bedrock measures of "c d- v orl ers ". 
The general idea is fiat rf a lot of people can perform many quick 
tasks, me sheer volu™ o~ consensus will balance tnei# lack of 
Individual eKpetise. But YilEhpul careful consideration for ti# 
dloe r sity of Raters, click-yvork turns into eKponeniial croupthink; 
baking cultural slase> directly mto training data. 







We try to label all the things 

Where did you acquire the knowledge 
necessary to successfully label Uiese data? 




We train the models 

How will the model be debugged? What 
do os H wrong’ look like? 




We recruit and test with users 


How will the backgrounds and fife experiences 
of your testers Influence your docisiors? 





r 



We interpret the user feedback 


What decisions are you trying to make? 
What evaluation metric will you prioritize? 




We craft the PR 

How will users see themselves reflected 
in marketing materials and demos? 




Finally, I’d like to offer 3 human-centered 
diagnostics when designing with ML. 

If you’re finding it tricky to answer these, It night be a 
signal to slow down and take a closer look. 
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If a human were to 
perform this task, what 
would ‘appropriate’ social 
behavior look like? 


What interpersonal cues might be relevant 
mat are missing trom your input or interface? 

E.g. body language, tone of voice. 


Tr-ko aiitocomplete for example: In what context would 
It he acceptable to finish another person’s sentence 
before theyVe stopped talking? 

Bear in mind that no one culture is capable of 

roproE anting urwore-al norms., especially for t&dal 

interactions, so we need to be mindful of our Intuitions. 
The psychological effects of algorithmic discrimination 

likely mirror those of social discrimination . 


autocomplete is an| ^ 

autocomplet e is an interruption 
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If a human were to 
perform this task, might 
we call it an expression of 
their personality? 

Tasks that have unambiguous utility, are 
perceived as repetitive or boring by users, 

anct'or benefit from super fast response times 
are ideal candidates for ML augmentation. 


Reply to all 


Grtsal idea! 


I like that idea. 


Pteasse like me. 


Grey area: Suggesting a reply In an email or SMS 

likely has a priming effect; impacting the user's; 
response even if they don't use it. And the fact that 
suggestions are even offered may lead the recipient 
to question the authenticity of the sentiment. 

Augmentation: The goals or a self-driving car are 
unambiguous, and the benefits of a computer's 
superhuman reaction time offer objective utility, 

Ng on? (hopefully) would say a driver got Into on 
accident because they were expressing the niseives. 
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Who are you? 

... OK, now what do your 
data teach you about 

everyone else ? 


Our traits don’t necessarily define us. but it’s foolish to 
pretend we don’t see them, By taking the potentially 
uncomfortable step of inventorying these traits— 
physical, social, cognitive, and otherwise—wo'ro 

getting proximate to those who are reminded of their 
differentness every time a 'default" is invoked in 
day-to-day life. 


I am.., 

Whte 


Male 


A parent 
Visually impaired 
Agnostic 
insured 

Urban-dwelling 
In my 30'=; 

■Cisgend^r 

yyv pgnceniile fielghi 


Affluent - 

Physical^ capable 
Culturally JevviBh 
ADome^w nef 
Nci college edu cnlad 
Married 
HdenaaeKual 
Inyoodm^nul ne^ldi 


A speaker of “standard* U.S. er*gi|$ft 
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Because hopefully that 
can help your world look 

a bit less like this... 


me 





everybody else 
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Because hopefully that 
can help your world look 
a bit less like this... 

...and a lot more like this 

















Appendix 




The goal of this section is to examine the 
disconnect between our expressed moral code 
and the unconscious biases that frame our 
perception and subsequent decisions. 



oi Gender roles 
in film 




GENDER DOLES IN FILM 


In 2015, women shared top billing with men in the 
top four grossing live-action films in the? U-S- 

How much were female characters 
seen and heard compared to men In 
those films? 











MAMIES IN FILM 


In 2015, women shared lop billing wiLh men In the 
top four grossing live-action films in tha U.S. 

female characters 
seen and heard compared to men 

those films? 
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awinnUMS tisHtntEon fciPidcr nWUMia 









EillOEfi ROLES IN FILM 


The numbers Improved slightly for 
the top grossing films featuring 
female leads. 

But still contain some surprisingly 
disproportionate numbers. 


if. 








BINDER ROLES IN FILM 


The numbers Improved slightly for 
the top grossing films featuring 

temale leads 

But still contain some surprisingly 
disproportionate numbers. 
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UNDO ROLES IN Fill! 


But the gap increased substantially 
for the top grossing films featuring 
male leads. 












31 NO! t HOLES IN HIM 


But the gap increased substantially 
for the top grossing films featuring 

male leads 


HUME 

arena Uawls hstHulppii Ciflnclar InModln 















EE NOH NOUS IN FUN 

Overall, for the top 100 grossing 
live-action films from 2015 in the U.S. 

naie ui Matters were seen Mai ntiara cuis we re hea ru 

1.84x 1.78x 


k mo r c- ih-o l female characters 


* more than female chambers 







U NDE1RIILEE IN HIM 


Related research 


In m ahing the to p 2$i 0 gro s t i ng filnnc 

if 201S in the LI S , women held 

19 % 

cf behind-the-scenes creative roles 


SOU IQS 

Speaker sex aid ;-:i inportienincnt cMaik. LuHcr and Scut 

Sander Inwitirlilv j" >ln*irrtlw Pfmclnarcn. Kninrw-siMmdE^raevt kiruEr 

arl~ind Itin-Sf Hnrn bnpkWTTJllLPlJBfcniFff c:n Hip I on 1 US. J W.niitl i.CO Hlint of an> Lnuipn 


'N ha n w om m and m pfi : pg g L :h p gif .art tarn p 

amount, women are pe'ceivej tu be speaking 

22 % 

we than men 


Wh on in p roPKQ d-gpn d pr g ro lj p. worn pfi 

speak less thammen until they comprise 

80 % 

of the groip 



One more observation 



Nave you ever noticed the “gender* 
makeup of Sesame Street? 







All ot the original starring monsters 
arc regarded with male aronouni, 
despite having no discernible 
physically gendered traits. 




It wasn't until 1993—24 years after 
the show started—that a female 
monster named "Zce H was added to 
the core cast of characters. 
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02 Skin tone in 
photography 





SKIN TCME IN mQTQGRAPHV 


This is a “Shirley Card 1 ’ 

Named after a Kodak studio model named Shirley 
Page, they were the primary method for 
calibrating color when processing film. 


5«JKES 

CulLn fi»n wdSliuH \u L^uufe Hmt'i wilfll I did LCdark-b^ni. iVlVi 

Now ttfldak'i ;Shlrley Curds 5eM Fhmxjra ^KN-Tcfip Silafiderd. NFS 






















SKIN TOME IN PHOTOGRAPH? 


Until about 1990, virtually all Shirley 
Cards featured Caucasian women. 


HUMES 
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HawFtiDlgpiapiiy Wia QmmuetJ rnr White Skin Caloi ^iGearKfTilcsl 




















TLHE IN FhDTDEnAPHV 


As a result photos featuring people 
with light skin looked fairly accurate. 


mmi 

■■■• r hi vt.i. L-jill V Hv.', -vl .jl "■ tM yjyik Sk ■ i. 

Cottui ima^ dud Cuyiiu^ Euufrv. Ruili 

HawFN^D^adwWjs OtitniiedlPi Whnr SklnealDi fF-lcepimniK:s;i 








SKIN TINE IHPtlDTO&RAPHV 


Photos featuring people with darker 
skin, not so much... 
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SKIN TONE IN FHDTQ EM PHY 


And when there was a mix, the 
difference was most noticeable. 
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SKIN TONE IN PHOTOGMPHV 


As society became more Integrated, 
photographers found workarounds. 

The most common techniques were to shoot with 
significantly brighter lights (making the room 
really hot!), using a stronger lash (42% brighter!), 
flnrl preparing separFitp ramprrss with riiffprpit 

calibrations for people with different skin tones. 


EDU EOS 

CalUurBalfiii^. hi^TeclYiQlofttt. and Etiufy. RflU 

hew Fiqkramlw'WaigpUnLKd Fvr 'rtTiiLc SKiri Mgr iRtewnumHF 














SKIN TOME IN PHOTOGGAPHV 


But motivation for innovation came 
from chocolatiers and wood furniture 
manufacturers. 

Kodak was receiving complaints that they weren't 
getting the right brown tones on chocolates, and 
that stains and wood grains were not true to life. 
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SKIN TDHE IN ?HQTOGRAPHV 


Related research 

As of l OR acres s all 2 m i 11 ion im plicil Ai&oc iation As u J ? M aero $ & h 11 u S. whi tel AT 

Test ClAT] participants participants.the median "D scare" was 

51 % 0.402 

showed a moderate to strong bias Tor white faces indicate a murle^le bias for white faces 
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As cf 7011, of aijils workirg in Engineering 
rales in theli.S. [ageIB and over) 

4 . 8 % 

were black 
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03 Automotive 
m safety 
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AUTOMOTIVE SAFETY 


Until 2011, female body-type crash 
test dummies were not required by 
the United States Department of 
Transportation. 
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AUTOMOTIVE S4FETY 


As a result, female drivers are at a 
higher risk behind the wheel. 

Odds 3 female driver will sustain severs injuries in an accident 

47 % 

4 higher than a riale driver 
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AUTOMOTIVE SAFETY 


Things are improving, but the target percentile for 
female test dummies remains problematic. 

Male body percentile Female bodypercEntile 


50th 5th 

STIKIDs WKBIUs 
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AUTOMOTIVE SAFETY 


Related research 




Scenarios, lesied usi ng ffema le bodily pc As of ?01S. the number of wo me n working inihe Ai of 2115, the numher ol car buying 

crash test dimmies in (he driver’s seat molar vehicle manufacturing industry decision influenced by women 


1 of 3 26.7% 
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We can’t remove human 
perception from the loop. 


And we can’t be gripped by 
inaction either. 




The inequity demonstrated in these examples may feel 
overwhelming, perhaps even a little disheartening. 

But we're in the right place at the right time and in 
the right industry to do something about it. 



